1. Field of the Description
The present description relates, in general, to techniques for determining a location of a mobile device such as a wireless or cellular phone, and, more particularly, to methods and systems (which could be labeled or thought of as an “audiolocate” method or an audiolocate system) that enhance localization of mobile devices such as phones (or phone navigation) through the combined and improved use of audio fingerprinting and audio watermarking.
2. Relevant Background
There are many instances where visitors to public and private venues, such as sport stadiums, theme parks, shopping malls, expos, fairs, and theaters, could benefit from mobile Location Based Services (LBS) provided on their mobile devices (e.g., smartphones and other wireless and/or cellular phones, tablets, digital cameras, handheld video games, and the like and the term “phones” is used herein as an example of such mobile devices). For example, LBS may provide visitors to these venues with relevant information associated with their present location in the venue to help them navigate through the venue, to organize their time and activities while at the venue, and to make their overall experience while visiting the venue more entertaining and enjoyable.
To be effectively implemented, LBS typically require that location information is available on a phone. To meet this need, a variety of localization techniques have been developed and utilized with varying levels of success. For example, localization technologies may rely on satellite (e.g., global positioning system (GPS) technologies) and terrestrial radio infrastructure (e.g., Wi-Fi and global system for mobile communications (GSM) based localization). This infrastructure or technology may not always be available (e.g., GPS is generally not available for use or effective when a phone is indoors) or may provide insufficient precision to provide localization with sufficient accuracy to facilitate providing LBS on a phone (e.g., GSM towers are sparely deployed). Further, these localization techniques make use of radio transceivers on phones, and radio transceivers are often energy-hungry devices that may quickly and undesirably drain the battery of a phone. Additionally, data transmission using GSM or Wi-Fi may not be “free” such that if using radio costs money it may be more desirable to use audio, which is typically free.
Audio-based localization in the form of either audio watermarking or audio fingerprinting has also been developed, but these technologies also have only been partially successful in supporting LBS on phones. Audio watermarking involves embedding imperceptible information into an audio content, such as music or a soundtrack, without affecting its audibility. The piece of embedded information (“audio watermarks”) can be received and decoded by a mobile device such as a smartphone. Audio watermarking enables location beacons to be embedded into the sound emitted by a loudspeaker. The beacons may simply contain the loudspeaker's unique identification (ID) and other information such as GPS coordinates and some contextual location information.
Since audio watermarking allows one to transmit different data with every individual loudspeaker, it enables phone localization on a per-speaker granularity. The location precision depends mainly on the density of the loudspeakers in a space as even if the speakers are playing the same audible content (as perceived by listeners) the embedded data can still be different. Hence, the location information is very precise. However, for audio watermarking to be applied in a real world environment, careful data embedding, such as with spectrum channel management to avoid interferences, and symbol synchronization are required. These have proven to be the limiting factors for the system reliability such that audio watermarking is often ineffective or at least less useful in noisy environments, e.g., environments where lower quality speakers are utilized, where multiple overlapping speakers are concurrently perceived by a phone, and where other sounds such as crowd noise are received with the output sound from the loudspeaker.
Audio fingerprinting enables an unknown audio sequence to be recognized by analyzing its perceptual characteristics and matching them against a database of known sequences. With audio fingerprinting, a mobile phone can be configured so as to be able to identify the audio content, e.g., a song, emitted from the loudspeakers. By relating the content to the location of the emitting loudspeakers, the phone (e.g., its processor(s) running one or more software programs or modules) is able to coarsely determine its location (e.g., the phone is within hearing distance of a particular loudspeaker).
Audio fingerprinting does not require any preprocessing of the audio content (if one does not consider creation of the actual fingerprint database), and it is more robust in noisy environments than audio watermarking. However, audio fingerprinting fails to provide the same high level of accuracy as is achievable with audio watermarking. For example, each loudspeaker in a venue space may not play or output distinguishable sounds. In fact, a number of speakers may be used to play the same sound or audio content, which in many localization or LBS environments can drastically limit the achievable location precision (e.g., the phone is located within a sound-perceiving radius of any of a number of speakers). Additionally, audio fingerprinting requires the use of a precompiled database of sequences for use in comparison processes to properly identify an audio fingerprint, and this database typically has to be stored in the phone's memory.
Hence, there remains a need for an improved method (and associated system) for providing improved localization of mobile devices such as phones. Preferably, such a method would be useful for providing audio-based localization and would be adapted for use within existing venues such as through use of existing sound systems and their loudspeakers, with audio-based location being fully independent from radio.